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The objectives of the Pell, Grant quality control (QC) 
System and the general definition of QC are considered. Attention is 
also directed tot the objectives of the Stage II Pell Grant QC system 
design and testing project, the approach used to develop the QC 
system, and the interface of the QC system and the Pell Grant 
delivery system. The comprehensive and strategic approaches to QC 
system development are compared us ing' the^ following objectives for 
the QC system as evaluative criteria: reduce error in the Pell Grant 
program, maintain flexibility to add additional programs, pilo^; test 
QC system components, and produce a sound QC methodology. The 
following QC options are assessed with attention to feasibility, 
pptential for reducing error, developmental costs, and interface with 
delivery system: a manual QC system with new data sources, a combined 
automated/manual QC system with, new data sources, a combined 
cuxtoraa ted /manual system with existing data sources, and a fully 
automated, integrated system. A combi ned , sy st#n wi th existing data 
sources is recommended. Appended are preliminary descriptions of QC 
subsystems that cover the Office of Student Financial Aid subsystem, 
the institutional subsystem, the Pell processor subsystem, and the 
student (applicant) subsystem. (SW) 
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. . \ E5CECUTIVE SUbAiAI^Y 

^ At ft February X2, 1982, nteeting between Division of Quality 

A^suffanqe personnel 'and Advftncei'.Technolc^ quality* control 

COC 3 project staff, issues facing the S^ge Twp study were 
'discussed. While tentative agreement w?is ' i^pached on th^se 
issues, DQA requested that Advanced Technology write a paper 
proposing i conceptual framewrk for the Stage Two QC system 
d^ign and tes.ting project. * In response. tb_Aat request, this 
paper considers s . ' ^ 

• , The objectives of the Pell ^ system and the general 

definition of pC ^ . ; 

, • / The pu^rposes of the Stage Two QC system design and ^ 
testing project 

^ * • ' Generic approaches to the development o^ the QC system 
y • » Alternative configurations for the Pell -QC |y stem 

.The latter two* areas— the a)^ro^ch used'to develop the. QC 

/ ' ^ . ft 

system and its ultimate conf iguratibn or interface with th^ Pell 

• : ■ ' ■ 4'. /' ' ^ i ■ * 

delivery system — are critical issues that need Immediate resolu- 
tion for the. Stage TWO" study to proceed. Based on a, detailed 
analysis of alternatives in both areas/ Advanced Technology 

recommends s * . ' ' 

♦ 

' • The use of a strategic and /modular approach to QC sys- 
' .0 tern development that will facilitate the incremental 
develojanent of ^ the n»st essential subsystems 
« ' . ^ 

• . Pending' completion of the functional analysis ih Stage 

Two, endorsement of a combined idfenual /automated QC" 
system^ that uses existing data sources to the extent 
possible ^ ^ 

. - . . ■ > " 



The implicatiqns of these reconanendations are .also con- 

• •*■««. ^ 

sidered for these procedural issues:. 

• The rationale for the Stage Two QC' study 
The desirability of a general QC. methodology 

• Key areas for QC subsystem developments 
Further discussi^ of these issues will be necessary as tbe 
Stage Two study pro^es«es\ , At the present time, it is essential 
that approval be giv^ for the interviews of ED personnel so that 
the Stage Two study can proceed. ' "5 • 
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PRODUCTION > ^ ' " • 

puaXity; cor^trol [QC3 systems are esseiySlal to tue sound 
management of .Federal" studeftt assistance programs. This ls> 
esseci^Xly trwe- for the Pell CSrant Program, one of ilie largest 
and m6st' prone/ to error and abuse. The annual overpayment 'in the 
Pell program, due to various institutional, student, ahd proces- 
sor error'A, is estimated to be in excess of $300 million. The 
development^ of a QC system for tli? Pell' program 'is complicated 
these factors , " " * 

• - The delivery system is subject to cliange as a. result of 
' administrative or legislative action. ' 

>i The current d el ive?7y system is already in operation and 
, involves ah extremely large number ^of primary actors. 

• Many dPbthe potential avenues for corrective action in 
the .current system only recently have been identified 
as a result of the gtage One QC study. >^ 



QC Objectives for the Pell Program ' ^ 

This paper proposes .a cofideptual "framework for the develop- 
ment <^ a QC system' for the Pell Grant Program. Recognizing the 
constraints now evident, the development objectives of the pro- 
posed CX: sys-^m are tos ' « • 



V 




Reduce error, fraud, and abuse in the current Pell 

Grant Program. • . 

- ' ■ ■ - # 

Maintain flexibility for adding new- components to the 
QC system and for adapting to changes in the Pell Grant' 
delivery system. 

Piiot test the QC system components in a^ reasonable 
time frame. , 

Develop sound system development metho dology for the 
Pen Grant Qb system. — ' — 
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Quality control, while essential to aliaost any eriterprise, 
is ambiguous and difficult to^ define pricisely because, as a con- 
cept, .i,t is applicai>le to such a wBfle range o£ activities. The 
essential element of a oc system; in eitlier an industrial or 
social service setting,, 4s the capability of detecting, prevent- 
ing, and correcting etrors or any tendency toward errors that 

occurs in an operational setting Impiicit in this definition is 

' • -. > ^ 

recognition that: " . 

• It is essen^tial to set standards for the output of a 
system. • 

\ e • standards must incorporate technical specifications 

presdribed by the provider of HOie service (or product) 
^ , and expected by J;he recipient ( or ^ consumer) . 
. • ' " ■ . . . ■ 

, • A system- for moni^ring standards and cor i:ec ting errors 
is necessary for the ia^lementation of an effective QCK 
system. . , ■ - 



The 'purpose of p. QC' system Is to increefse the effi- 
ciency and effectiveness of the delivery system by 
reducing costly erijors a'nd increasing the "quality of 
services (or products). ; 

, ^ . . • , , . X 

THE STAGE TWO QC S^DY ' ' ' ' ' ^ I. 

During Stage One of -the QC study. Advanced Technology 
examined the. applicabilitfy of quality control concepts- and pro- 

ctedur«S"tto student financial assistance programs, in order to 

' ' , • . ^ , V ^ - \. ' - 

develop a OC cc/hpept for the Pell progifa^. The project teamr 



Reviewed relevant literature ptj industrial and 
governmental oc c^y^tems and practices ' 

Interviewed QC person|fel from five social service 
agencies .operating at the Federal, state,' and local 
levels ->'■•■ 



• Cqnaucted a workshop/discussion personnel 
^ and ED stktf. . 

Purposes of the Stud'gint* Aid 6c System 

\/' ^ ^ ... ■ . - ' ' • ' 

From Stage One ^ctivities, a working concept of quality con- 
trol wa^ developed foj^ student financial assistance programsV 
parti-cularly the Pell Grant Prograio. it is es%enti4l that a QC 
system for student assistance programs .enable ED to: 

Develop stmndkrds and measures for monitoring the 
delivery- of . student aid. 

Measure performance of student aid del^ery against 
specified laeasures. . 

' ■' . . ^ * 

Determine ^and monitor errors in eligibility determina- 
tion and awar^ prdces'sing. • . 

Identify sources and probable causes of errorfe to t>lan 
corrective actions. ' - ' ^ 

Develop corrective action proQedures .as an -integral 
part of the processing functions. 

Develop standards and measuree for monitoring the^ ^ 
^eslilts of corrective actions. 

Ensure. that various actors (e.g., processor s or* insti- 
tutions)* are operating in accordance with specified 
procedures/ regulations, and standard^. 

Report appropriate Qd, information to ED personnel on a 
timely basis. % 

This will make it necessary for ED to develop QC procedures 
performed on an ^ongoing basis as the award process is taking- 
placVyC front end) and on, a audit ba%is. aft^r the award process 
has bee^i completed ^ack endK P/>r the Pell Grant Program, it is 
desirable that front-end procedures be developed to reduce the 

. .." * / ' . 

amount of error prior to the' disbursemej^t of funds. It ife also . 
necessary to have a back-end con^onent to eiisure that these 



• cox^rections ^ave actually taken place,, as well as to iferforra 
analysis and develop corrective actions tor future years* 
Conceptual Model for the (K! System^ ' . 

^ At this early time it is not possible to provide a detailed 
conceptual model of thf Pell QC system. - since this is. an objec- 
' tive pf jthe Stage Two siudy. However/ it is po^ible to concep- - 
tua^ize the basic components of such a systera. Conceptually, a 
Qd system foi* the Pell program would combine a core OC Management 
Information System COCMISj for monitoring and trat-Xing the award 
process with a series of subsystems for the major actors in the 
awa^i process s the processor, OSFA (Central and Regional ) insti- 
tutions, and studentsl r The subsystems i(«>uld both monitor perfor- 
mance against standards and determine error levels within, each 
feubsystaB. A simplified conceptual iwidel is illustrated in 
Figure 1 . , - * 

Steps in guilding a QC System , ' 

. In practice there could- be raore^d^^a these f pur subsystems . 
Each subsysteri must interface .with both ;iianual and automated 
systems and therefore jnust combine elements of bath. The devel- 
opment of a QC system for the Pell program will- be a complex 
process requiring 1 \' - / t^ ' -^ i * 

,• / petailed analysis o^the Pell delivery system 

• Develojmient of atyConc^jotttaX ijiodel for an overall QC 
isystem - r, 

#1 Systematic" A^velojaaent of sul^ystems that integrate 
' ' into the delivery syste^Q 

,' ' ^ " ' ^ 

• Development of a central monitoring and tracking sys- 
tem, 1^)>e QC* Management Information System [QCMIS 3 
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GENERIC APPROACHES TO ^ SYSTEM DEVELOPMENT 

T6 set the stage for ED decisions kbout the design and 
developn^nt of the Pell Grant Quality Control System, it is 
important to consider two generic approaches to QC system devel- 
opment. Both approaches are applicable to procedures already 
deveJoped. The first is characterized as the comprehensive 
approach, the second as the sttateqic or modular approach. ^ They 
are summarized and compared in the following sections. . 
Comprehensive Approach , ' ' ' 

The conqorehensive approach assuiKies that anything that can go 
wrong with the operating system will go wong? therefore*, it is. 
in^KJrtant to* identify every possible error in the system, and 
design corrective j>rocedures. The basic steps involved in this- 
approach are outlined as follows (with the application of each , ^ 
step to the Pell Grant Program considered in parentheses.) . 

• Identify the major subsystems _of the programs (for 
•Pell Grants this- would include^ eligibility determina- 
tion, award processing, fund disbursements, and so 

' forth). 

• Identify the major actors or cpmponents -for each sub- 
system (for. the .eligibility subsystem this would 

^ . include students/parents, institutions, , Pell proces- 
sors ) . ^ • ' : 

• For each actor in each subeystem, identify acceptable 
standards of possible errors in the system (for stu- 
dents and parents in thd eligibility system subsystem 

' this will include identification of staifeards for 
application errors).' / 

• Define measures for each set of standards- (this %*ould 
; • include identification of data elements and procedures 

for information collection). 
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Determine the components of each subsystem that merit 
development and inclusion in the QC ^stem {evaluate 
the f easibilitYt of including the various information 
sources in a <iC data base). j ' ■ • 



• Proceed with sye±:gm -develojwient -en the select subsystem 
(design and develop procedures? for implementatibiv 5f 
selected coii^»onent& of each, subs ystam) . ' 

♦ . 

• . • ■ 

For th« Pell Grant Program, the con^rehensive approach 
would result ^n an elaborate QC system that imposed new data col- 
lection procedures on top of the already existing system. It 

would permit the developnent of .a sejparate/or stand-alone QCMIS. 

-» ' . ' 

could also be used to produce QC manuals for training ED - 

" ' ' , ■ . *^ ■ ■■ 

personnel (Centr'al and Regional) and institutional representa^' 

tives in the practice o'f QC in the Pell Grant and other student 

ai4 programs . To the extend th^t the system, used automated -data 

■ ' . . ■ f ' * • 

collection and analysis procedures, it would be labor intensive 
since an entirely new set of procedures wojuld be neede^for each 
component of each subsystentr and sophisticated data base manage- 
roent procedure's rnay be needed centrally, depending on how rauch^f 
the syetem is eventually implemented. ' * Vj- 

Strategic Approach ° - - * 

This approaeh assumes that the major sources of ,error in the 
Pell Graijt Program can be identified and that corrective action 
should be made, in these areas through the developnent of nu5dular 
eca^xonents oJ»|j^ overall ^ system. The basic steps involvW in 
the stra^i^^^^fcjach are outlined as follows (with applications* 



to the l>ell*^rWI Program) 
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• p Identify iajor sources of error in the program (for the 

Peil Granft Frogra© this was, accomplished in the Stage 
One <XJ study ) . • ^ ^ 

• Conduct a fuhctional analysis of the operating system, 
including information requirements, linHage structures, 
and breakdown points {identify the places in the system 
where corrective action should be talcen and ^nitor ' 
progress) * ' * V ' . ** 

• Conceptualize a QC system wi.th modular ccaiqpdnent^s 
designed to correct and monitor error prone functions 
{conceptual- system design would consider modular sub- 
systems that interface ,the QCMIS with subsystems of the 
Peil Grant system, e.g,, processor, PIMS) . 

m ' Select and prioritize modular! subsystems for- develop- 
roen^t identify time :irame for specifications, design, 
^ . d*>^^k>ment, and installation of each selected sub- 

yf*^Sp* . ^ 

• Proceed with. systema,^evelopment for selected subsys- " 
terns (e,g., develop, procedures and'^systetas manuals, 
user manuals, syst^ specifications, and software spec*^ 

. ifications, as necessary)* 

m Perform System tests on mcsflular subi^stems ^S;, they are* 
completed (this would be -determined as .the work sched- 
ule wa^s .developed and approved> . 

For the Pell Grant Program, the strategic approach to QC 
development 'would permit the incremental implementation and test- 
ing of modular QC subsystems designed specifically to reduce 
errors in the system. The nodular approach could also permit the 
use of up-to-date electronic technolc^ utilizing preexisting 
data sources, Where appropriate, rather than developing new data 
sources and reporting procedures. In other instances 'it might 
result in more systematic' analysis and . reportii^ on data cur- 
rently reported. - The principal advantage of the itwdular design 
Is the use of preexisting data sourC?es with enhanced, reporting 



_ani} inoisitoriitia capabilities* However, in some areas (e.g., 
, application It^ error) <3«ta sources %#ould be needed. 



■ Evaluat ion • . • . 

X ~" — ' : • . ^ ♦ ' 

THisv section coi^ares the cbmpr^ensive and str'ategic 
aii^roaclies to QC^system^ development using the following objec- 
tives o£ the projiosed QC syst^ as evaltiative criteria. » 
Reduce Error in the Pell Grant Pfogram • • 
• >' Maintain Flexibility to Add Mditional Programs 
^ • • Pilot Test QC System Components^ 
• * Produce Sound QC Methodology 

The two approaches vary in their potential for reducing 

errolrs , particularly the Qverpaymehts identified iia^ the OQ study 

stage One.. The costprehensive approach would^ build in an elalx>- 
■ . ^ . ■ . ■ ■ . • • 

rate proce4ure for identifying all possible\error/but woul^ also 

introduce con^licating factors such as possible overeniphasis on 

less important issues and time delays in monitoring Key factors 

/ 

e to the implementa|:ion and operation of new data collection 
procedures. a?his would severely constrain the in^lementation of 
front*end. corrective actions. The strategic approach jwould be 
targeted at critical problems in the design stager it wuld also 
atten^t to use, wherever possible, preexisting data sources. 
This would ensure that systems develojanent worK %<!buld be targeted 
at the, most critical problems and minimize the time delays 
required in obtaining timely information. ' This approach could 
also be oriented toward front-end and back -end correction* 



#' • 



Therefore, the strategic ai«pro«ch would naxiraize the potent;ial 

for reducing profram ^osts. 

' ' " ■ ' ■ ''^ 

/ The two approaches also^vary consiaerably ih their flexi- 

t , ■ 

bility tg» adg aaditionaX pgograras to the QC system.^ With the 

comprehensive, system it is possibXe to ifepen^t the same proce- 
; ■ . . * : . ..... ■ 

dure for other student aid programs or to modify the systtem and 

data correction procedures to include other student aid programs. 

^ Iii contrast, the strategic approach could be targeted at whatever 

parts of the itudent aid system ED decided to add. It would 1^ 

possible/ depending on the conceptual Resign finally selected, 

either, to create new modular subsystems to an overall integrated 

QC syst«a or to create, new parallel systems for the otber pro- 

gr^s* In fact, the design of new subsystems could be staged to 

coincide with changes in the delivery system. 

The strategic ai^roach %«>uld also provide ED with the oppor- 
tunity to pilot test QC system- Gogg>onents at hn earlier stage in 
the process. Inherent in the c«^prehensive aj^roach is the 
design of jiew data collection procedures that involves defining 
liew data elements and devising new collection procedures, %?hich 
takes time to implement and test. The strategic approach empha- 
sizes the use of ejcisting data sources, a time saving procedti;re, 
and the phased development of QC subsystems. This should allow, 
for pilot testing components of the QC .system earlier in the 
development process. 

Bqth approaches would produce sound QC methodologies hut 
i«oul4 have different results with respect to the type of 
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methodology develo|^d and used. The eos^rehensive approach would 
re9^It in the same methodology^ beipg applied to each subsyst^ of 
the Ptell program, with critical subsystem components receivihg 

more attention tHan less critical ohes«*# Since is an elaborate 

. , ' , " . *• . . 

system Review process » an elaborate and Well--de£ined procedurfe* ^ 

♦ '.^ ■ ■ ' ■ *^ 

vfould be' necessary. It is possible to apply the same procedure 

to other student aid prt^rams. The strategic approach wuld have 

■ ■ ■■. ■ ■■ «»- * 

to use sound system development procedures . in the condeptual 
developcaent o£ the.QC system and moddlar subsystems. ^ As a part 
of the overall functional analysis, the contractor wo^ld evaluate 
thfe value of the QC procedures already in use # for example. An -• 
manuals currently distributed to institutions, before developing 
new procedures. . Therefore^ the procedures developed as & result 
of the project would be specifically oriented toward the existing 

• • " t ■ ■ ; ■ #■ 

-system, but the procedures used for the entire project %«>uld be 

applicable to all Title IV programs. 

. ' - *...,'". . . 

In svimmary, -given the objectives of the system, design and 

testing phase of the Stage Two QC study Vsf the Pell Grant Pro- 

/ ■ . ■ ■ / . V 

gram, the strategic* approach to systems development increases the 
prospect oft 

f ■ 

• 'Reducing errors in the PeYl program 

• r Adding new Title IV programs to theQC system 

• Pilot testing QC components earlier in the study 
' • , ^ Developing a sound generalized methodology 

At a redent meet^'g (^n the QC syst^ design, DQA officials 
expressed a desire to achieve these ends* Advanced Technology . 
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eoDOttrs with ^is emphasis and recoiemends thaTt the strategic * 
approach used in the Stage Two study. The. -approach can be 
^siXy adapted to the August 1981 worlc statement .( see Appendix 

4 

'■ ■ ■ ' • : . ' 

jkLJERHJlTIVS QC SYSTEM a>liFIgtlR&TXOKS . . ' 

Wifi^e it is neither possibly nor desirable at this early 

.- -» 

v^tage to provide a detailed description of what the QC system v 

will actually, include, since this is an objective of the QC 
. system design and testing project now underway, it is . importartt 

* * ■ • ■ 

to spedulate about What the QC system might look like and how it 
will interface with the Pell delivery system. It is particularly 
difficult, witfiout undertaking the functional analjfcis of the 
Pftll delivery system, to ppecify the extent to which the QCMIS 
will be a manual or /automated system and the extent to which it 
will use new or existing data souses. Two issues help distin- 
guish between optional conf igurations^or the 00 -Systems 

' - ^ 

• The extent to %rtiich the new system is manual or 
automated 

^ • The use of new br existing data sources 
These issues can be used to generate four discrete configuration 
options. The criteria used to evaluate these options arei 

• Feasibility of the' system design (Can it be done?) 

• Potential for redujging error . (Will it save money?) 
* • Developmental costs ( How much? ) 

• Interface' with the delivery system (Will it work?) 
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At this point, consideration of the actual configuration of 
the QC system is speculative* However, it. is possit»Xe to outline 
ana c<Mipare a few options, using these as evaluative criteria.. 
.As ^ part of the systm development pro ject, a rigoroifts and 
aetail,ed analysis is necessary.. At this time, however, a brief 
analysis is called for to point out the types of decisions ED 

will confroij(t in a relatively shorfperiod of time. 

^ ■ - ■ - . i' ^ - 

Option. 1? Manual Enhancements ; ' ^ ' ■ 

It is possible to develop a QC .system that essentially^ pro- 
, ■ ' ■ /' ■ • 

vides manual enhancements to the existing Pell delivery, system. 

In this ease the system development activity vKJuld include 

If 

detailed specification of repor^d^ requirements for each of the 



major actors in the Pe|l Grain: ^ogram. The reports would be 
entered into a filing system in the Division of Quality Assurance 
and used as a means of monitoring aKd tracking progress on cer- * 
tain Xey areas identified' during the s^tem development process.^ 
Either the comprehensive or the strategic**4pproach to the system 
develojanent (outlined at^ve) could be used to develop the manual 
enhancements, although the comprehensive approach is easiest to 
adapt to this option. The major problems with this option are 
that it would add to the reporting burden of the major actors and 
that excessive time delays would be built into the system. 

* . . ♦ . 

Option 2 8 Combined Manual /Automated QC System Using Sew Data 
Source^ T T""*^ ' ' 

This option would essentially take the above^stated concept 
an additional step. Where appropriate data colleqteJd through 
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■> * . , ■ ' . ■ r - ■ • 

process- could be automated, some file«^W^uld be accessible for 
automated analysis and reporting. . Others,' patrtioularly periodic 
summary Reports usir^ aggregated data, would remain manual \ Com- 

, parked to Option iV. this option would go an extra step, using \ 
automated technology as appropriate %i7ithin -the framework of cr«- 
ating a new data collection system. It has the f>otential of prb- 

vVid^ing data on a someWhat more timely basis, ^ but it would provide 
an additional layer of reiiorting on top of the existing delivery 
system. It would use either system development approach but is 
more adaptable to tlie comprehensiSre approach. 

' ■ . • 

Option 38 Combined Manual/Automated QC System p^ing Existinq 
Data^ sources" ~ 7= ~ . * ^ ' ^ 

• f " . . 

This option would ha^e some of the same features as Option 2 

but would emphasise .^n^ lysis of existing data sources in ?»ew ways 
rather than develojanent of entirely new data collection proce- 
dun»s^ This would involve utilizing. Where appyopyiate, data the 
major actors currently generate |gr that could be generated rela- 
tively easily through modififcation of current practice. In sc»ne 
instances this might involve reporting progress in corrective 

actions and in the speed of handling certain types of problems. 

* ' ' ♦ * * . 

For example, if the IRS Form !• 040 cross-check were built into the 

Pell processor contract, a system for monitoring and reporting on 

this activity could be built in as, well. This ajmroach would be 

flexible enough to add other student assistance programs as 

necessary. In this way a QCMIS subsystem could constructed 

tAat dealt with critital points in the delivery system. This 
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option would be ii»st adaptable to tl^e strategic approach to QC 
*^eystem^deve!lopra«ftt. ^ ^ 

Option 4; Fuliy /^tomated Integrated QC System U ainq Existing 
Data Sourcgs , • - — r-* r =■ 

• » - ■ * * 

^ This option wc>nXd taXe the approach ju&t^dentified to the 
fullest possible degree of system autopiatioa . Such 2m option 
would integrate the core QCMIS with systems that interface with 
the major actors' operating systems. Monitoring, cc^aring 
results to specific standards, taking routine corrective actions, 
and reporting could be built into the 'system. Such an approach 

would be dependent on automation of most compor^ts'df the Pelf 

t. . . ' " , ^ \ * 

^ , ystera. Other ^tudent a i<5 pro-ams c6vild be- integrated 

into th« system, depending on Vhether and how the student assis- 
taince delivery system was eventually redesi^ed. This option ^ 
could use _ a variation of either the strategic or the comprehen- ^ 
sive ai^oach to systems development I>ut would have to be done in 
ccwibination with the redesign of the entire delivery system. 
Assessment • « , , 

A functional analysis of the Pell Grant system is required 
to fully evaluate th^se options. In the absence of such an anal- 
ysis, only preliminary judgments are* considered. Ctoce the funo* 
tional analysis of the prograirf is complete, it will -be possible 
to develop these and/ or other options in greater detail, in which 
case it will.be possi^e to provide a detailed evaluation. Table 
1 pUBBnarizes the preliminary assessment of the four generalized 
options. This assessment uses the evaluation criteria just 
identified . • ^ 



DesctPlptiOR 



Manual QC System with 
Hew 9ata^ Sources 



islMlity 



Modlrate (Can be 
Impleme'nted wlth exls^ 
ting delivery system— 
mamia! analysis may 
take too long to . 
useful . ) ' 



tow (Delays b&llt Into 
rijforting Hffllt nionl- 
torfng front-end cor- 
rective action. 
Impjeinentatlon may not 
lead th error reduc- - 
tlon.) 



DevelofHnental , Moderate (Requi res 
Costs developing entirely 

new systetii. ) 



Potential- for 
Reducing Error 



Interface With 

Delivery 

Syst«i 



Not 



Integrated (Result 



tegi , 
creation of* neSr, 
parallel system.) 



PREIIHINAAY ASSESSHEHT OF 
QC SYSTEM OPTIOMS 



OPTION 2^- 

Combined Aytomated/ 
ttonual QC System with 
'New Dat^ Soupces 



H1^ (Can he In^le- 
Tnented with existing 
delivery system;) 



Moderate (Delays 
built Into reporting 
Jlmlt monitoring 
front-end corrective 
action.) 



High (Relmlres devel 
oping entirely new 
system. ) 



( 



Not Inte 
■{^sult 
of new, 
system. ) 



rated 



s creation 
parallel 



OPTION 3 



^ Combined Automated/ 
Manual .QC System 
with Existing Data 
Sources 



High (Can be Imple- 
mented with existing 
delivery system sys- 
t^.) • . 



< 



High (Integrated 
approach permits 
monitoring front-end 
corrective action.) 



jtqderatg (Uses exls- 
tlng data to the 
efxtent possible.) 



Partially Integrated 
(tlesult Is Integra- 
tlon of (K^ subsyst^ 
with Pell delivery . 
system. ) 



OPTION 4 



Fully- Automated, Inte 
grated QC System 



Low (Requires delivery 
sjitem redesign.) 



High (Integrated 
approach permits inonl/- 
torlng froiit-end 
recti ve action.) 



c^- 



Uriknown (Included in 
delivery system rede^ 
sign which would be 
costly.) 



Fully Integrated 
(Result Is full 1 nte- 
gratlon of QC into 
Pell delivery systera.^ 



On the basis of this pteliialnary assessment it is possibia 
to ^e an iiiitiai judgioent about QC system opt4on is most 

desirable. Option 1 wouia probably have relatively imxSe^jb " ^ 
result^, ifirith respect to its potential for ^educing error, and . 
moderate developmental costs? it %.(puld not be integrated w'itli the, 

dtelivery system. Joption 2 would increase the potentTal for 

L ^ ' ' 
reducing error anp the^ developmental costs. Option 3 has 'high ' 

potential for reducing error/ would require moderate develop-^ 
^mental costs con^jared to Options 2 ^ and yould partially, 
integrated into the delivery system. Option 4;^4|iil^\iaying tbe,: 
highest potential for ^reducing error, does not- apjpeaQf easible at 
*th* present time.. As t^e Stage Twd QC stu<iy progJfe^e4 / hewever, 
it could be further evaluated as part of the deli v^y system 
redesign. Of the four options. Option 3, a corolSined automated/ 
i^nual system using existing data sources* appears to be the most 
desirable QC delivery system configuration. ' 

Whilfe^ the preliminary, assessment indicated that Cation 3 is 
the desirable configuration for the Pell Grant -QC, a full evalua- 
tion of which configur-ation is i^st desirable will not be pos- 
sible until the functional analysis is con?>lete at the end of 
TasX 5. However, it appears that a cfombined manual/automated QC 
system using existing data sources i'^ desirable. It^ is- also 

highl^^ compatible wit^ the strategic approach to QC system 

. . - * • V 

development. 



CORCIiOSfOW'S AND IMPLICATIONS ^ ' \^ • ^ y 

On: the basis of thife analysis, it appears that: 

^ # The strategic e^proach to OC syst^ development should 
be .ttsed to design and test the PelX Grant QC system. 

/ - • ' ■* , \ 

• Based on a preliminary assessment, a coxnbi.'ned auto*- 
. mated/iiianuar QC Configuration using existing data 

, sources %#ould be the s^ndest system design 

alternative. ' , - . ■ 

■/..•'. ^ • 

It is 'also iu^rtant to consider tbe in^jli cations of these 
findings for the desigi^ and develojsnent of a QC system for the 
Pell Grant Program. Thi^e other practical issues confront the 
project: • 

« 

• ^ The rationale lor the Stage Two study* 

• The develofwnent of a generalized OC methodology' 

f Determination of which QC subsystem should have the 
higfhest priority for developoent 

Th^se isfeues have emerged as a result*"©^ the February .12, 

1982, discussions between Advanced Technology and DQA. The fol- 

lowing i^ a revi^ew of their conclusions. 

Raypnale for QC System Development 



I >mi'le existing docuii»nts establish the parameters for the QC 

system design and testing of taie Stage Two QC project* e^speciially 
the worX statement for Stage Two, there is a need for greater 
' 'Clarity about the rationale for these activities. Two factors 
c^tribute to this issuer ' 

• Recent changes in the Department of Education relative 
to the eventual redesign of the student financial 
assistance delivery system 

• ^ The recent coii?>letion and dissemination of the ^tage 

One report on corrective actions to the Pell Grant • 
delivery system 
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There appears to be a pelativfl^ high pifobability of change 
^in^the student financieU. assistance delivery system in the not- 
too-distant f*uture"^ Recently five s^^ig^^el task forces were 
initiated by>ED to examine major a'^pec^^^plbie student fioancial 
aid ,d^Xivety system, including quality control atfd de3^ivery sys- 
teai redesign. , While the current* Pell Grant delivery, system is 
not lilc^ly to undergo major revision in .th€J short terSf,' there is 
a^significant possibility for m£nbr changes in the ^hort term amd 
major changes in the medifpa term, i.e.', three to five years* To 
the extent possible under the exi string contract, the Advanced 
Technology project t€sam is interested in naadifying' the system 
design and testing part of the «€age Two QC to meet the-changirtg 
n€feds of ED. This will require^ flexibility in the system devel- 
opment approach used during Stage Two, an issue considered 
earlier, as w^ll\as recognition that the QC project now faces a 
different situation than it did a year and a half ago When the 
pr'oject began. Our assumptions abqut how this situation will 
influence the QC study arei 

• The QC system should be designed for the current Pell 

, delivery system. Which should remain relatively stable 
during the ne|Et three to five years. • 

• The QC system design project should* develop and test 
subsystems on a pribrity basis rat}ier*than test the 
entire system at the end of the project'. 

• Subsystems can^ be developed after the conceptual desig 
f phase of the pro-ject is completed. 

• The conceptual design for the QC syst^n should maintain 
flexibility to adapt to a changing Pell Grant delivery 
system. 
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• Hie QC system design, to the extent possible^, should be 
I adaptable to t|ie eveentual i-nclusion pf other Title IV 

I a)s84.8t2u^e programs. - 

# The QC system can be viewed as an effort to develop and 
>ilot test some 'quality control and corrective action 



\ methodologies thfit could event^lly be incorporated 
. . into a new studejst assistance delivery system. 

Stage. One of the -C^ality Control Study idsfttified a wide 
range of recbmmefndatipns . for 'corrective aqtions in the Pell Grant 
delivery system. I^e report, considered .both mechaBical actio ns' 
to make marginal changes in the current delivery system and major 
structura l changes in the way Pell grants and other Federal aid 
are delivered to students. These changes were considered fort 
• The applicant and application cott^joneht 
' • The instittJtional ccsnponent ^ 
. • Processor component 

To the ^extent possible, the conceptual design of the QC 
system should facilitate the implementation of these recoam^nda- 
tions, especially. the mechanical actions. The major struc- 
tural changes are subject to policy deliberations in the Depart- 
ment and therefore cannot be anticipated, itowever, the QC system 
design effort #111 maintain flexibility relative to tHese and 
other structural changes. ' 

These interrelated factors — the prospect for change in the 
student aid delivery system, and the recent completion of the 
Stage One QC study recommending corrective actions — provide a 
major input to our ongoing deliberation with ED about the QC 
system development activit^ies. A primary concern of the project 
team is that the deliverables developed during the project be 
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Weiettl to the De|>artji^elii Of E^^ not call for an exhau&tiv^ 

Effort to, djpveiop a -full-blown syst^a that will Iwi^e a ehdrt life 

.- :\ " • ■ • , ^. • • ' ■■■ ' ^ 

'c^nce it ier iiap|.^nted. This situation is a Xey factor in our. 

effott %p-::k4hpt':'^pe T»«> of the project to tseet the changing 

n#ds^ qf the De^rtJAent. 

, General Methodol ogy- ' 



The methodological aspects of the system development phass 
of Stage Two are critical to the success of the project. In 
fact< the methodology component could he o^e of the roost in^r- 
teujt projects of Stage Two. In (Jisctissions on Stage Two* DQA 
personnel expressed interest in a general QC methodology that 
could be used for trainiug purposes and adapted to ' diverse organ- 
isational settings. A large number of actors, inclfiiaing laany of 
the approxiraately 6,000 institutions, need, to improve QC prac- 
tices. A general 'qc methodology could be disseminated through 
the jpSFA/DQA. • 

Ac^vaneed Technology can expl9re the possibility of and need 
for a generalized QC methodology as part^f the functional anal- 
ysis. It will be neOessary to review docuipents new disseminated 
to institutions to evaluate whether QC methodology is already 
covered 4p these documents. If not, t^en the systems design 
activities could be modified to include this. 

One viable strategy for achieving this would be to adapt a 
proven MIS planning and evaluation methodolbgy to a general QC" 
methodology. Detailed specification could be for Specific 
organizational settings, ranging from large universities with 
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automated financial aid systems to saall. Institutions with manual 
systems. The top-doum approach to MIS planning and evaluation* 
pioneered originally l3y IBM, has been widely adapted to >!IS plan- 
ning problems for both manual and au'tcaoatea information systems. 
This method emphasizes s . 

• Top-dcwn MIS planning with thV possibility of bottofc-up 
implementation * • 

• Viewing data as a resource and an information system as 
a sup|«>rt fat functional processes-^ ' 

• A proven, structure4 methodology for MIS assessment and 
evaluation - ^ - _ ^ 

The combined top^own, bottom-up technique is highly com- 
patible with the modular af^oaoh considered earlier. The metho- 
dology requires adherence tef a procedure lAioh generally includes 
the following six steps, the actual elem^ts of each varying 
according to which adaptation of ^^fie procedure is used, ^ 
Survey of current systems 

Functional analysis of management requirements 

♦Sj^stem conceptualisation . 

" f' ■ ' - ' ., 

System design 

System 'installation , 
If the strategic approach is adopted, then these basic pro- 
cedural steps will be used by Advanced Technology in the OC 
design and testing activity. A general isaed methodology fcAsed on 
this procedure. could be developed, ^ith appropriate instruments- * 
tion an4 detailed specifications foif different organizational 
environnmerits . Ci|rre?itly the project is entering the functional 
analysis ate^ As a part of this step, Advanced Technology can 
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.eyalii&te the need for a generaliaed aethodolc^; if ED con- 
siaered dt desirable to have siasli a ipeth^sdology, it couid be 
developed as part o*£ the system conceptuaj-ization plxaee. This ' 
issue will reguire More discussion Ijetveea Advaiicea Technology 
and the ^ ' 

Key Areas for Subsystem Develoi«ttent 

. At this time it is difficult to move beyond the initial con- 
ceptualization of the QC systm (Figure 1), since the funciional 
analysis has not been coa^leted and the system conceptualiza- 
tion has not yet been initiated^ However/ it is important to add 

r ■ ■ . ■ ■ ■ 

^Ome initial iiafy^ssions about %^at might be include^^ in the 

system tos 

• Provide a £rame\<orlE - for the system development process . 

, Provide .p. basis for discussion between Advanced 

Technology and the DQA about priorities for system 
development. . 

As the earlier discussion indicated, the Pell QC system 
could have four basic subsystems s 

• OSFA subsystem 

• Institutional subs^tem 

• Application processor subsystem 

• Student (aj^licant) subsystem 

It is important to facilitate a dialogue about system- 
"development priorities. A brief description of what" might bje 
included in each subsystem is included in Appendix IJ. These 
descriptions will no doubt l?e modified as the jaro^ect evolves. 
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gtimmary 

This paper has identified and analysed several decisive 
issues that nejed clarification and resolution. Two of these 
issues mst be resolved in order to proceed with the QC system 
design. Their resolution will help provide a framework for this 
activity. Ba^ed on this analysis. Advanced Tecihnolpgy ' 
reeomaenda-s v ^ , ^* 

• ^ A strategic apptoa<^ for the development of the QC 

system ■ , - 

• Pending the con^Jletion of the functional analysis^ a 
ccxnhined raanual/automated QC system using t,6xisting^ data 
sources " 

■ ■ . ■ ■ ' ■ \ ' ■ ■■ ' ■ 

ProiSedural issues are also raised and addressed in this 

paper. While definitive^' answers are not required at this time, 
dialogue about them can clarify the stage Two ssfstaa developsnent 
effort. The ifsues are; 

_ • ' changing rationale for the Stags Two study 

• ; The development of a genera^liaed Qd methodology 

■ ■ * 

• Priorities for QC subsystem development ^ 

^ In this. Section we have suggested a framework for resolving 
these issues and hdpe that our analysis wiai help "focus future 
discussiops about the QC study 

• • 1 
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. APPSS&IX I 
QC SYST®! DELlVERABLES/SCHEKJIiE 



1. 

4. 

♦5. 
*6. 
7. 

8. 
* 11* 

*i2 

*13. 



^nag ement ofijectives and QC ' 
Staadards 

• 1 

Operatioz^X Eiiyi^cmii^nt and 
• Resource? Report 

Potential QC Linkage with OtJier 
Title iv Prograias 

Functional Reguireiaents and 
Conceptual Design 

System/Subsystefe Specification 

Preliminary Implementation Plaft 

I>ata Base Specifications/Data • 
Elentent Dictionary I - 

- . . - ^ ' 

Interface Rfsquireinents and 
Specif Iteationd - 

Detailed Specification and Test 
Plan, 

Detailed S<t^lementation/lnstalla' 
tion Scl^edule 

Conversion Plan 

/Draft Processor User Manual 

^Pinal User Manuals . 



DATE DUE ^ 
March 15, 1982 



April 16, 198^ 
July 31 V 1982 



May 31, 1982 

August 2, 1982 
\ August 16, 1^2 
September 1, 1982 

September 15, 1982 

October IS, 1^82 

November 15, 1982 

November 15, 1982 
December 8, 1982 
Decembex 31, 1983 



*If strategic, approach is selected by OSPA, the Deliver- 
ables/Schedule will be based on OSPA component pritsrities. 
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APPENDIX II 
PRELIMINARY QC SUBSYSTEM* DESCRIPTIONS 

OSFA Subsystem ' ^ 

1- 'Control of funds for the Pell Grant Program, as well as the 
aceoui^ting of institutional expenditures « is accoasplished through 
the Pell Grant Program Information and Monitoring System [PIMS]. 
This system is thd responsibility of the Division of Program 
operations Cdpo]. Funds are initially authorized (budgeted) to 
. individual institutions through the annual initialization pro- 
ceap. Institutions request modifications to the authorization 
via progress repor^^ si^jfamitted each October, February, and June. 
As needed, institut-ions'subrait.ad hoc pit%ress reports. These/ 
reports also include exp^ditures to -date Which are processed by 
PIMS. Student Eligibility Reports CSERs*! for individual recip- 
ients ar 6 submitted with the progrejss reports an^processed 
through the PIMS SER subsystem. Funds are disbursed to insftitu-* 
tions via EDFMIS and EDPMTS systems after validation of institu- 
tions' req|jest8 against PIMS authorization level. 

The 0SFA| subsystem to the QCMIS could integrate quality con- 
trol procedures on these processes into the Pell Grant Quality 
Control System after discussion with DPO personnel. Possible 
automated Quality* Control procedures includes 

• *-^Early warning reports" When overall institutional 

authorizations exceed stated percentages of the program 
■ budget ' - ■ *r ■% 
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• **^rlY warning repcjrts" ^en individual inst;itution 
expenditures exceed stated percentages of. the^ autlior- 
ization ' , 

• Monthly reconciliation of PIMS expejtiditures with the 
disburseroeht syste^ 

• timeHne^s audits of PIMS proce^tng 

fm Co%ari8on of PIMS SEH subsystem data with processor 
data ♦ 

Institutional Subsystem ,/ 

OSFA currently jaonl tors institutions -.through the Division of 
Certification and Program Review t'DCPRj. This "division conducts 
Program Reviews and' revieitfs financial Aid Audits of Postseeondary 
Institutions, The audits are performed by e*ither independent 
auditors, institutibnal 'internal' auditors, 4>r state auditors. A 
comprehensive audit guide far the Pell Grant Program has 'been 
developed jointly by the American Institute of "Certified Public 
Accountants EAICPA3 and ED. All post6econdary institutions 
participating in the Pell Grant program ^e required to ^ have a 
biennial audit covering t%*o years or annual audits. Results of 
these audits are forwarded to ED's 10 regional of flees- for 
jreyiew. ' *^ r 

These monitoring procedures are back-end qyality control 
procedures that could be integrated into the Pell Grant Quality' 
Control System. Our strategy for integration of these existing 
prbcedur|^^eg ins with discussions with DCPR and selected ED 
Regional offices. These discussipns will enable the project team 
analysts to fully understand the procedures and existing infonaat 
tion flow. We could then recommend modifications tos 
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• Enhance the existing program review process , as 
* . required. 

• ttodify existing ia£oni»tion flow -to integrate Program 
Review results into the Pell Grant Quality Control 
System. 

• Develop ipforraation flow to iijtegrate results of Finan- 
cial Aid Audits ii\to the Pell Grant Quality Control 
System. • 

Additionally, the Pell Grant Quality Control Syst^a could 

eventually provide OSPA withi , / . 

\ ' 

• A "front-end" warning list of institutions that have 
not ^bbmitted audits ' v / 

• Selection of institutions for program reviW, based on 
error prone prof iJLing ^ 

Pell Processor Subsystem ^ - . 

Whfie the processor does not ^ontrit»ite significantly to the 

errors in the Pell program, there is room for it^tovement in t^fi^s 

aspect of the^ delivery system. The roFe of the processor comp^ 

nent of the*.QC syst^ is complicated by unresolved issues such as 

the possibility of including an IRS check for income and possible 

family suf^rt in the processor contractor. In spite of these 

uncertainties, it is possible to specify ' areas of the proces-' 

s^r functions that can be reviewed in order toi ' ■ 

, m -Develop an automated processor sut^ystem as part of the 
QC system. 

• Develop new QC specifications for the Pell processor 
contract. ^ 

These issues deserve exploration as they relate to the 
automated and manual functions of the Pell process. For Example, 
the automated subsystem could includes 



A ajretem for testing ana monitoring al X |>ossible errors 
in the £»:ocessor 8yst:em * 

• ^ -^' ^--'^ ■ ' ' ' 

^ • 'RQutine reports on the number of aj>^lications and the 
time .ifequired to issvie SERs to students 

» . ■ ^ . ' ■■. . ' . . 

' • Boutine reports : to identify students matchina error 
prone profiles ' 

^ Several hundred people are involved in the processing of 
$»ell applications. Advanced Technology could examine the possi- 
bili$<y of introducing stricter ^QC procedures in ^«iis^ function. 
This could result in a mechanism for monitoring the cost of the 
processor contract Which might be incorporated into the RFP for 

* \ 'i 0 

the processor contracC- The essential elferoents might be: 

• Clear definition of every step of application proces- 
sing and development of detailed instructions for each 
{contingency 

• . . Identification of controls for each' ke^i step to ensure ^ 

applications ar^ not lost, misplaced/ or delayed / 

• Elimination of duplicaton of effort by streamlining the 
• . process ^ ' ' ' 

• Setting of time, standards for mailing SERs and estab- 
' lishing^ monitoring, procedures 

Student (Applicant) Subsystem 

The student subsystem is the most difficult to conceptualise 
without conducting the functional analysis. In practice, the 
subsystem will probably involve a direct interface between the 
processor system and the PIMS to ensure, to the extents- tKJssible, 
funds art disbursed to students (through their institutions) in a 
proper manner. These could be handled through routine reports 
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^rom the cixietlng eystem or the development of an automated sub- 
syst^n that would, perform a variety of cross-checlcs of student 
files. * 

It is possible to create a student subsystem to monitoz; 
front-end corrective actions by the Pell processor and perform 
bacK-end follow-throu^' with PIMS and ADS disbursement. Such a 
system would be especially useful if the IRS check were built 
into the Pell contract. If this were the case it would be pos- 
sible for the student sut^iystem to incorporate the following 

Develop dual data efttry of the Pell ap|>lication and the 
IRS 1040 (if submitted with application) " 

Develop separate edit checks on both^files to reduce 
key entrx erroff 

Jteichine chec^ for IRS match with SER * 

■ « 

Monitor IRS <5h^ck with routine summary re^rts to OS^k 
(could b^ transmitted electronically) , • 

Create a QC student file jwith corrected SER information 
(this could be done by DOA) ^ 

Match of the QC student file with the PIMS SER subsys- 
tem, to check flagged students 

Eventually incorporate back»end review and audit of the 
student Hie with the current validation procedure ■ 
( could be done by ED' Regional offi^j;) 

A student subsystem generated by the award processor and 

cro^s-checked with PIMS could potentially provide a system for 

routinely monitoring exyrors and identifying areas for corrective 

action. It might also use data ^f reus other sourcesi such as the , 

Social Security Administration, to enhance the potential of the 

QC syst^ to identify errors. 
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